NIST AI 600-1: register the framework and transcribe its 211 suggested actions - #187
Conversation
…gistry (refs GenAI-Security-Project#119) Signed-off-by: Prasad Deshpande <74897556+Prasad-desh@users.noreply.github.com>
Signed-off-by: Prasad Deshpande <74897556+Prasad-desh@users.noreply.github.com>
Signed-off-by: Prasad Deshpande <74897556+Prasad-desh@users.noreply.github.com>
Signed-off-by: Prasad Deshpande <74897556+Prasad-desh@users.noreply.github.com>
Signed-off-by: Prasad Deshpande <74897556+Prasad-desh@users.noreply.github.com>
Signed-off-by: Prasad Deshpande <74897556+Prasad-desh@users.noreply.github.com>
|
Thanks, @Prasad-desh. This is careful work, and the notes for reviewers made it quick to check. Verification of this branch (
One fix needed: 13 descriptions keep a line-break artifact. Where a hyphenated compound wrapped at the hyphen, the line break became a space, so the text reads
Six of the derived titles carry it too ( One claim to correct: 14 The Left for the maintainer (my view in brackets, not a ruling): the title-derivation rule (verbatim text stays in Merge-order note. This PR and #184 both change CI hasn't run here. Workflows on fork PRs wait for a maintainer to approve them, so every result above is from a local run, not a GitHub check. |
|
Following up with the maintainer's decisions, so you know exactly what's needed to merge. The four calls you asked about are settled your way. The title-derivation rule stands, the 49 subcategories stay out of this registry, the stub mapping files stay out, and the branch name is fine. Needed before merge:
Also landing separately: #188. The webapp home page counted every registry as a framework, so this PR would have shown 27 frameworks on the live site while the README says 26 mapped, and added 211 unmapped controls to the headline. #188 makes the home page count mapped frameworks only. It doesn't touch your files, and with it merged, this registry can land without changing any public number until the first rows map to it. After merge, a framework-owner reviewer will author the mapping rows (per #119 and STRAT-04). Thanks again. The transcription itself checked out to the word. |
|
got it @emmanuelgjr , will do the changes. thanks for the updates |
Signed-off-by: Prasad Deshpande <74897556+Prasad-desh@users.noreply.github.com>
…rent claim Signed-off-by: Prasad Deshpande <74897556+Prasad-desh@users.noreply.github.com>
Signed-off-by: Prasad Deshpande <74897556+Prasad-desh@users.noreply.github.com>
Signed-off-by: Prasad Deshpande <74897556+Prasad-desh@users.noreply.github.com>
Signed-off-by: Prasad Deshpande <74897556+Prasad-desh@users.noreply.github.com>
Signed-off-by: Prasad Deshpande <74897556+Prasad-desh@users.noreply.github.com>
|
@emmanuelgjr all three done, rebased onto 1. Hyphenation. Fixed, including the six derived titles, which are regenerated from the corrected text rather than patched. The cause was mine: the extraction joined words across a line break without rejoining compounds at the hyphen. The parser now does, and a 2. Parent claim. Reworded rather than adding the subcategories, so the AI RMF control count is untouched. The 3. Two things the risk column turned up that need your call:
Verification on the rebased branch: Understood on #188 landing first. Thanks for checking all 211 against the PDF; the two things you caught were both mine. |
The home page added every registry to the framework list, mapped or not, and summed every registry's controls into "Registry Controls". A registry that lands before its rows (NIST AI 600-1 in #187) would have shown 27 frameworks against the README's 26 mapped, and added 211 unmapped controls (+19%). Frameworks now come from the mapping rows alone, and the controls total only counts registries a row cites. No change to today's numbers: all 26 registries are mapped. New webapp test pins the hero to stats.json's frameworks.mapped. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Prasad Deshpande <74897556+Prasad-desh@users.noreply.github.com>
|
Thanks, @Prasad-desh. All three changes check out, and I verified the new risk tags independently. Verification of
Your two calls: I'd keep both as you have them.
From the review side this is ready. Next, the maintainer approves the fork CI run, and the PR merges once the checks are green. |
What this PR changes
Adds NIST AI 600-1 (the AI RMF Generative AI Profile) as a framework registry. Refs #119 — the agent-safe half of that ticket: registration and transcription only. No mapping rows are asserted, and no vulnerability IDs are affected. Which suggested action addresses which entry is security judgment (C4) and belongs to a framework-owner reviewer per STRAT-04.
data/frameworks/nist-ai-600-1.json— 211 suggested actions across 49 AI RMF subcategories.control_idis the document's own Action ID,descriptionthe Suggested Action verbatim,titlederived mechanically from it,gai_risksthe document's own GAI Risks column.inventory_completeness: complete, 211 of 211data/framework-sources.json— registered,current_version: "2024-07"scripts/control-ids.js— id grammar^(?:GV|MP|MS|MG)-\d{1,2}\.\d{1,2}-\d{3}$, socheckControlIdShapes()covers it from day onescripts/control-ids.test.mjs— four tests: the grammar, all 211 ids against it, and everygai_risksvalue against the document's twelve risksscripts/exports.test.mjs— see Notes for reviewersdata/stats.json,docs/frameworks-registry.js, README markersOn
parent: it is the AI RMF subcategory the action is filed under. 45 of the 49 resolve againstdata/frameworks/nist-ai-rmf.json; MG-4.3, MP-3.4, MS-2.12 and MS-2.13 do not, because that registry holds only the subcategories existing mappings cite. The$commentsays so.Type of change
Source / evidence
NIST AI 600-1, Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile, July 2024 — https://doi.org/10.6028/NIST.AI.600-1
Every control id, description and risk tag is transcribed from the suggested-action tables of that document.
Checklist
Content
data/frameworks/*.jsonLinks & data
.mdlinks resolve — no.mdfiles changeddata/schema.jsoncompatible — the registry validates againstdata/framework-schema.json;gai_risksis an extra property, which that schema permitsProject hygiene
changelogblock; no mapping file was modifiedCHANGELOG.mdupdated — conditional on a new mapping file, which this isn't. Happy to add an[Unreleased]note if you'd preferREADME.mdcounts updated — file count unchanged; the freshness marker moved 4 current → 5 vianpm run statsFor new mapping files only
N/A — no mapping file is added.
Notes for reviewers
Verification, rebased onto
mainat 633c59a.npm test93/93. A clean clone of that base is 89/89, so the four added tests pass and nothing regresses.validate.jsreports 0 errors and 91 warnings, identical to that clean clone.npm run buildis deterministic across two runs, andgit statusshows only the intended files.Changes made in response to review.
/\w- \w/search over the file returns nothing.parentclaim is corrected in both the$commentand the description above, naming the four subcategories that are not innist-ai-rmf.json. Rewording rather than adding them, so the AI RMF control count is untouched.gai_risksadded for all 211 actions, transcribed from the GAI Risks column, with a test asserting every value is one of the twelve and that all twelve are exercised.Two things found while transcribing the risk column, both worth a decision.
The tables spell four risks differently from the enumeration in section 2:
Harmful Bias and Homogenization(57 rows),Environmental(4),CBRN Information and Capability(2) andHuman AI Configuration(1). The registry normalises to the section 2 names, since those define the twelve and the test checks against them. The variants are recorded in the$comment. Say the word if you would rather keep the table spellings and widen the test.GV-1.4-002 carries a fifth tag,
Civil Rights violations, which is not one of the twelve and appears nowhere else in the document. It is not recorded ingai_risksand is noted in the$commentinstead of being silently dropped or silently admitted.Why
exports.test.mjschanged.prose-shaped control ids do not spread beyond the known setruns an OSCAL export for every file indata/frameworks/, butcompliance-report.jsresolves--frameworkfrom the names the mappings cite, so a registry nothing maps is not found. The fix skips those, exempting nothing that does produce a prose id.One transcription detail. A pattern match over the PDF finds 212 ids, but
GV-1.1-002appears only in the explanation of the Action ID scheme. GOVERN 1.1 has exactly one action; the tables define 211. Recorded ininventory_completeness.Merge conflict resolved.
data/stats.jsonconflicted with #184 as you predicted. The branch now mergesmainand the file is regenerated, so the control total is 1184 (973 after #184, plus these 211).Transcription and verification done with AI assistance; I reviewed the diff and ran the build, validators and test suite locally.